Information display terminal, information display system, and information display method

ABSTRACT

In a wearable information display terminal, information related to an object is displayed at a timing required by a user in an easily recognized form. Photography is performed in a field-of-vision direction, a first object is detected from a first image obtained by the photography, relevant information related to the first object is acquired from a network, and a second image related to the relevant information is generated. A second object different from the first object is detected from the first image, the second object is used as a display trigger, and the second image is displayed when the second object is close to the first object.

TECHNICAL FIELD

The present invention relates to an information display terminal, aninformation display system, and an information display method.

BACKGROUND ART

Various kinds of methods have been proposed as an information displaymethod related to a so-called wearable information display terminal thatdisplays predetermined information using a goggle-type or glasses-typehead mounted display.

For example, Patent Document 1 discloses a method of switching aninformation display mode between a summary mode and a detailed modeaccording to a situation of a user. Further, for example, PatentDocument 2 discloses a method of combining information input by the userthrough an input unit with information photographed by an imaging unitand displaying combined information according to a situation of a realworld specified by a current position or object information.

CITATION LIST Patent Document

Patent Document 1: JP 2007-163634 A

Patent Document 2: JP 2011-28763 A

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

A difference between a glasses-type wearable information displayterminal and a mobile terminal such as a smartphone lies in that, whenpredetermined information is acquired, in the mobile terminal, the usershould actively takes a look at display content provided by the mobileterminal, whereas in the wearable information display terminal, the usercan confirm information display passively within a field of vision whenit is in an operation state. However, in the wearable informationdisplay terminal, since the information display starts and endsregardless of the will of the user, particularly, the informationdisplay is performed even when unnecessary, and thus the user may feelannoyed according to circumstances.

In the technique of the related art disclosed in Patent Document 1, theinformation display mode is switched according to a situation such aswhether the user is walking, viewing a display video, or speaking buthow to determine a timing at which the user desires information relatedto an arbitrary object to be displayed and how to start or end thedisplay of the information related to the object were not particularlytaken into account.

In the technique of the related art disclosed in Patent Document 2, theinformation related to the object input by the user is combined with theinformation photographed by the imaging unit, the combined informationis displayed, but similarly, how to start or end the display of theinformation related to the object at an effective timing was notparticularly taken into account.

The present invention was made in light of the foregoing, and it is anobject of the present invention to provide an information displayterminal and an information display method, which are capable ofstarting or ending display of information related to an object at anappropriate timing and displaying the information in a form easilyrecognized by the user in a wearable information display terminal or thelike.

Solutions to Problems

The object is achieved, for example, by the inventions stated in claims.

Effects of the Invention

According to the present invention, there is an effect that display ofinformation related to an object can be started or ended at anappropriate timing, display information can be displayed in a formeasily recognized by the user, and convenience of the user in, forexample, the wearable information display terminal can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of a configuration ofan information display terminal.

FIG. 2 is a sketch diagram illustrating an example in which a user wearsan information display terminal.

FIG. 3 is an operation flowchart according to a first embodiment.

FIG. 4A is a first diagram illustrating an example of relevantinformation display according to the first embodiment.

FIG. 4B is a second diagram illustrating an example of relevantinformation display according to the first embodiment.

FIG. 5 is a block diagram illustrating a system configuration on anetwork.

FIG. 6 is an operation flowchart according to a second embodiment.

FIG. 7 is a diagram illustrating a display example at the time ofdisplay trigger registration.

FIG. 8A is a first diagram illustrating an example of relevantinformation display according to the second embodiment.

FIG. 8B is a second diagram illustrating an example of relevantinformation display according to the second embodiment.

FIG. 9 is an operation flowchart according to a third embodiment.

FIG. 10A is a first diagram illustrating an example of relevantinformation display according to the third embodiment.

FIG. 10B is a second diagram illustrating an example of relevantinformation display according to the third embodiment.

FIG. 11 is an operation flowchart according to a fourth embodiment.

FIG. 12A is a first diagram illustrating an example of relevantinformation display according to the fourth embodiment.

FIG. 12B is a second diagram illustrating an example of relevantinformation display according to the fourth embodiment.

MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described withreference to the appended drawings.

First Embodiment

FIG. 1 is a block diagram illustrating an example of a configuration ofan information display terminal 101. The first embodiment will bedescribed using this.

101 denotes an information display terminal, 102 denotes a control unitthat controls the information display terminal 101 in general, 103denotes an imaging unit, 104 denotes an image processing unit, 105denotes a display control unit, 106 denotes an audio input/output unitcorresponding to a microphone or an earphone, 107 denotes an audioprocessing unit, 108 denotes a communication processing unit, 109denotes a communication input/output unit, 110 denotes a positiondetecting unit including a global positioning system (GPS), 111 denotesa sensor input/output unit, and 112 denotes an various kinds of sensors.

The communication input/output unit 109 is connected with a networkthrough wireless communication. In the present embodiment, an example inwhich the information display terminal 101 is connected directly to theInternet or the like and acquires information will be described.However, the information display terminal 101 may perform communicationwith an information terminal (for example, a smartphone) serving as aseparate device through near filed communication (NFC) such as Bluetooth(a registered trademark), and the information terminal may perform aconnection with the Internet or the like and a main process.

The information display terminal 101 is a head mounted display equippedwith at least the imaging unit 103 and the display control unit 105. Asvarious kinds of sensors 112, a plurality of sensors such as aninclination sensor that detects a posture of the user, an accelerationsensor that detects a motion of the user, a visual line sensor or atemperature sensor that detects a body state of the user, and a pressuresensor or a capacitance sensor used as an input/output interface (I/F)that detects an instruction input of the user can be mounted.

FIG. 2 is a sketch diagram illustrating an example in which the userwears the information display terminal 101. Components that are commonto those of FIG. 1 are denoted by the same reference numerals.

201 denotes the user, 202 schematically indicates an area of a firstimage captured by the imaging unit 103, and 203 schematically indicatesan area of a second image displayed through the display control unit105. The area of the second image is not limited to a size or a rangeindicated by 203 and may have a size or a range capable of covering afield of vision widely. The information display terminal 101 can haveany form as long as it is worn on the head, and the display control unit105 that projects, for example, light according to information withinthe field of vision of the user 201 and displays information and theimaging unit 103 capable of imaging the field-of-vision direction of theuser 201 are arranged. For example, the information display terminal 101may be a glasses-type information display terminal, a goggle-typeinformation display terminal, an information display terminal worn onone eye, or an information display terminal worn on both eyes and is notlimited to the exemplary form illustrated in FIG. 2.

The image processing unit 104 analyzes the first image of the area 202photographed by the imaging unit 103 and detects a predetermined objectincluded in the first image. Information indicating a feature of thedetected object, for example, information indicating a feature relatedto a pattern which will be described later is transmitted to the networkthrough the communication processing unit 108, and the informationrelated to the object accordingly acquired from the network is suppliedto the image processing unit 104. The image processing unit 104generates the second image based on the information, and supplies thesecond image to the display control unit 105. The display control unit105 causes the second image to be displayed on the area 203.

For example, the display control unit 105 causes an image to bedisplayed on a half mirror installed at a position corresponding to thearea 203 or displays the second image by projecting the second imagedirectly onto a retina of the user 201, but when the information displayterminal 101 is of a transmissive type as at least a head mounteddisplay, and the second image is not displayed, the area 203 is set notto block the field of vision of the user 201.

FIG. 3 is an operation flowchart according to the first embodiment.

FIGS. 4A and 4B are diagrams illustrating an example of relevantinformation display according to the first embodiment, and illustratethe field of vision of the user.

An operation of the information display terminal 101 according to thepresent embodiment will be described in detail with reference to FIG. 3and FIGS. 4A and 4B. The present embodiment will be described inconnection with an example in which, particularly, when the user shopsaround for products in stores or the like, the information displayterminal 101 displays relevant information related to a predeterminedobject as the second image.

First, in step 301, a process starts. The whole process may be initiatedby a touch input of the pressure sensor in a sensor 112 or a voice inputof the user using the audio input/output unit 106. The process may startautomatically when the user is recognized to stay in a store based onposition information detected by the position detecting unit 110 or animage photographed by the imaging unit 103.

Then, in step 302, a predetermined object is detected. Specifically, theimage processing unit 104 detects a predetermined object by performingimage recognition on images consecutively photographed by the imagingunit 103 and performing matching between a plurality of clipped objectsand a shape or a color of a predetermined pattern or a transformprediction value thereof. The predetermined pattern may be a defaultpattern (for example, a genre such as a “suit”) according to storeinformation or may be acquired from the Internet or the like inassociation with the store information.

When a desired production is decided, the user may designate a productname, a model number, a genre, and the like in advance, a predeterminedpattern may be acquired from a server on the Internet or other publicinformation, and predetermined object detection may be more specificallyperformed. The number of predetermined objects detected in step 302 isnot limited to one, and a plurality of objects may be simultaneouslydetected in an imaging area.

When the predetermined object is detected, in step 303, relevantinformation about the predetermined object is acquired. Specifically,the communication processing unit 108 acquires information related tothe predetermined object from the Internet or the like through thecommunication input/output unit 109. When the product name, the modelnumber, and the like can be decided on based on a product tag or thelike from an image of the predetermined object photographed by theimaging unit 103, the relevant information can be also acquired based onthe product name, the model number, and the like.

Then, in step 304, determination of a display trigger is performed. Thedisplay trigger is a specific object for initiating the display of thesecond image based on the relevant information, and in this case, a“hand” is used as the display trigger. When the predetermined object isdetected in step 302, although the relevant information of thepredetermined object is acquired in step 303, the display of the secondimage is not performed unless the display trigger is photographedthrough the imaging unit 103 and recognized through the image processingunit 104. A method of recognizing a “hand” through the image processingunit 104 is the same method as the object recognition in step 302.

For example, as illustrated in FIG. 4A, although 402 to 405 in animaging area 401 are detected as a predetermined object, since there isnot display trigger in the imaging area 401, the display of the secondimage is not performed (No in step 304).

On the other hand, as illustrated in FIG. 4B, when a specific object 407serving as the display trigger is recognized in the imaging area 401 anddetermined to approach a predetermined object (Yes in step 304), in step305, the second image based on the relevant information of the specifiedpredetermined object 402 is generated through the image processing unit104. Further, the second image is displayed through the display controlunit 105 in a form that can be recognized as information related to thepredetermined object 402.

In this case, content displayed as the second image 406 is productinformation related to the predetermined object 402. Here, for example,reviews information or market price information related to the productare displayed in the form recognized by the user. The displayed contentis preferably information useful for the user and not particularlylimited. A broken line of the imaging area 401 is illustrated for thesake of convenience and not displayed within the field of vision of theuser.

In step 306, the determination of the display trigger is continuouslyperformed, and the relevant information is displayed while the displaytrigger is being recognized.

Further, when the position of the display trigger is changed, and apredetermined object is changed, replacing to a second image related toa changed predetermined object is performed (Yes in step 306). When thespecific object 40 serving as the display trigger is hardly recognizedin the imaging area 401, in step 307, the display of the second imageends, and the process returns to the detection of the predeterminedobject in step 302 (No in step 306).

FIG. 5 is a block diagram illustrating a system configuration on anetwork. The acquisition of the relevant information through thecommunication processing unit 108 will be briefly described withreference to FIG. 5.

The information display terminal 101 acquires the relevant informationof the predetermined object 402 from the Internet 501 through theabove-described configuration. For example, unspecified third partiestransmit the information via the Internet 501 using a personal computer502, a mobile terminal 503, an information display terminal 504, and thelike, and the information is accumulated in a storage 506 by a server505. The information display terminal 101 transmits informationidentifying the predetermined object 402 detected from the imagephotographed by the imaging unit 103, and the server 505 suppliesinformation related to the predetermined object 402 among theinformation accumulated in the storage 506 to the information displayterminal 101.

The server 505 collects information about predetermined stores,predetermined regions, and the like in advance, and supplies theinformation to the information display terminal. However, when aspecific server 505 has no collected relevant information, theinformation display terminal 101 may searches for and acquireinformation related to the predetermined object 402 from informationdisclosed by a plurality of servers connected to the Internet.

As described above, according to the first embodiment of the presentinvention, the wearable information display terminal starts or ends thedisplay of the second image for the relevant information of thepredetermined object, for example, using the display trigger of causingthe hand to approach the object. Thus, the display of the second imagecan be performed at a timing and a period of time in which a will oraction of the user (for example, an action of picking up a product in ahand=a will desiring to see information related to a product) isreflected. The user can use both hands freely in the state in which theinformation display terminal is worn and need not operate theinformation display terminal 101 to select an object, and thusconvenience is improved. Since a situation in which the second image isdisplayed at an unnecessary timing or a situation in which informationabout a plurality of objects is displayed simultaneously does nothappen, there is an effect that a display annoying the user can beprevented.

Second Embodiment

Next, a second embodiment of the information display terminal 101 willbe described with reference to FIGS. 6 to 8B. A configuration of theinformation display terminal 101 is basically the same as thatillustrated in FIG. 1, and thus the description will proceed using thereference numerals allocated to the respective components of FIG. 1.Similarly to the first embodiment, the description will proceed with anexample in which, when the user shops around a product in a store or thelike, the information display terminal 101 displays the second image asrelevant information about a predetermined object.

FIG. 6 is an operation flowchart according to the second embodiment.

FIG. 7 is a diagram illustrating a display example at the time ofdisplay trigger registration.

One of features of the present embodiment lies in that the user canregister the display trigger. FIG. 6 illustrates an operation flow atthe time of display trigger registration, and FIG. 7 illustrates thefield of vision of the user at the time of display trigger registration.

First, in step 601, the process starts. The process may be initiatedaccording to a user instruction given by a touch input of the pressuresensor or a voice input. Then, in step 602, a frame in which the displaytrigger is registered is displayed. Specifically, the image processingunit 104 generates an image indicating a registration frame, and thedisplay control unit 105 causes a registration frame 702 to be displayedwithin a range of an imaging area 701 of the imaging unit 103, that is,in the field-of-vision direction of the user as illustrated in FIG. 7.

In step 603, the user positions a specific object that is desired to beregistered as the display trigger to fall within the registration frame702. In this case, a “palm” is registered as the specified object 703.Then, in step 604, when the image processing unit 104 recognizes thatthere is the specified object 703 within the range of the registrationframe 702 from the image photographed through the imaging unit 103, aninquiry indicating whether or not the object is registered as thedisplay trigger is given to the user. The inquiry given to the user maybe a text display as illustrated in FIG. 7 or may be a voice output fromthe audio input/output unit 106.

When a registration instruction is given by an instruction input of theuser (Yes in step 604), the specified object 703 is stored in a memory(not illustrated) in the information display terminal 101 as the displaytrigger. The memory is installed to be accessible from the control unit102 or the image processing unit 104 but may be installed in the controlunit 102 or the image processing unit 104. Thereafter, in step 606, theprocess ends. Further, in step 604, in a state in which the presence ofthe specified object 703 is recognized by the image processing unit 104,an image indicating a contour of the specified object 703 may begenerated and displayed by the display control unit 105, and the usermay be notified of the fact that the specified object 703 has beenrecognized through a sound output from the audio input/output unit 106.

FIGS. 8A and 8B are diagrams illustrating an example of the relevantinformation display according to the second embodiment. An example inwhich, when the user shops around a product in a store or the like, theinformation display terminal 101 displays the second image as relevantinformation about a predetermined object, similarly to the firstembodiment will be described with reference to FIGS. 8A and 8B. Here, a“palm” is assumed to have been registered in step 605 as the specifiedobject 703.

The operation flow is basically the same as that of FIG. 3, and thus adetailed description thereof is omitted. In the determination of thedisplay trigger of step 304, as illustrated in FIG. 8A, there is apredetermined object 802 in an imaging area 801, and the relevantinformation is acquired from the Internet or the like, but since aspecific object 803 is a “back of hand” and not recognized as theregistered display trigger, the display of the relevant information isnot performed. The image processing unit 104 performs the determinationas to whether or not the specific object is identical to the registereddisplay trigger using matching of the shape or the color of the object,the transform prediction value thereof, or the like. In FIG. 8B, aspecific object 804 is a “palm” and determined to be the display triggerregistered in step 605 of FIG. 6.

In step 305, the image processing unit 104 generates the relevantinformation of the predetermined object 802 specified by the displaytrigger as a second image 805. The display control unit 105 causes thesecond image 805 to be displayed in the form that can be recognized asinformation related to the predetermined object 802.

Similarly to the above description, content displayed as the secondimage 805 is preferably product information that relates to thepredetermined object 802 and useful for the user and not particularlylimited. Broken lines of the imaging area 701 and 801 are illustratedfor the sake of convenience and not displayed within the field of visionof the user.

As described above, according to the second embodiment of the presentinvention, a specific object is registered as the display trigger in thewearable information display terminal, and thus it is possible to startor end the display of the second image according to the user'sintention. Similarly to the first embodiment, the user can use bothhands freely in the state in which the information display terminal isworn and need not operate the information display terminal 101 to selectan object. The display trigger can be appropriately changed according tothe user's desire, and thus convenience is improved.

As a modified example of the present embodiment, an embodiment in whichthe process is initiated by the registered display trigger in step 301of FIG. 3 is considered. In other words, as an operation of theinformation display terminal 101, first, in step 301, only thedetermination of the display trigger is continuously performed. Then,when the object recognized by the image processing unit 104 from theimage photographed through the imaging unit 103 is determined to beidentical to the registered display trigger, the operations of step 302and subsequent steps are performed. In the example of the presentembodiment, a series of operations can be performed by showing a “palm”in the imaging area 401. Since the start of the operation is simple,there is an effect that the convenience of the user can be furtherimproved.

Third Embodiment

Next, a third embodiment of the information display terminal 101 will bedescribed with reference to FIG. 9 and FIGS. 10A and 10B. Aconfiguration of the information display terminal 101 is basically thesame as that illustrated in FIG. 1, and thus the description willproceed using the reference numerals allocated to the respectivecomponents of FIG. 1. In the present embodiment, similarly to the firstembodiment, the description will proceed with an example in which, whenthe user shops around a product in a store or the like, the informationdisplay terminal 101 displays the second image as relevant informationabout a predetermined object.

FIG. 9 is an operation flowchart according to the third embodiment.

FIGS. 10A and 10B are diagrams illustrating an example of the relevantinformation display according to the third embodiment, and illustratethe field of vision of the user.

Operations of step 901 to step 903 are the same as the operations ofstep 301 to step 303 in FIG. 3, and a detailed description thereof isomitted. When the process starts in step 901, in step 902, apredetermined object is detected, and in step 903, the relevantinformation related to the predetermined object is acquired from theInternet or the like. The number of predetermined objects detected instep 902 is not limited to one, and a plurality of objects may besimultaneously detected in an imaging area.

Then, in step 904, a visual line of the user is determined. A gazingpoint of the visual line of the user wearing the information displayterminal 101 is detected through the visual line sensor included in thesensor 112. For the visual line sensor, for example, a method usinginfrared rays disclosed in Patent Document 1, a method using acomplementary metal-oxide semiconductor (CMOS) sensor, and the like areknown. When the gazing point falls within a range to which the detectedpredetermined object pertains (No in step 904), the display of thesecond image related to the relevant information is not performed. Forexample, as illustrated in FIG. 10A, although 1002 to 1005 in an imagingarea 1001 are detected as the predetermined object, when a gazing point1006 falls at none of the predetermined objects, the display of thesecond image related to the relevant information is not performed.

On the other hand, as illustrated in FIG. 10B, when the gazing point1006 falls at a predetermined object 1002, and this state is continuedduring a predetermined period of time (for example, 3 seconds) or more(Yes in step 904), in step 905, the image processing unit 104 generatesa second image 1007 as the relevant information of the predeterminedobject 1002, and the display control unit 105 causes the second image1007 to be displayed in the form that can be recognized as informationrelated to the predetermined object 1002. Similarly to the aboveembodiments, content displayed as the second image 1007 is the productinformation related to the predetermined object 1002, and for example,the reviews information or the market price information related to theproduct is displayed in the form easily recognized by the user.

The displayed content is preferably information useful for the user andnot particularly limited. A broken line of the imaging area 1001 and thegazing point 1006 are illustrated for the sake of convenience and notdisplayed within the field of vision of the user. However, a mode inwhich an image is displayed as the mark of the gazing point 1006 may beprovided so that the mark of the gazing point 1006 can be recognized bythe user, and in this case, the shape of the mark is not limited to theshape of the gazing point 1006.

In step 906, the determination of the visual line is continuouslyperformed, and the second image 1007 is displayed while the gazing point1006 falls within the range to which the predetermined object 1002 orthe second image 1007 pertains (Yes in step 906). It is because thegazing point 1006 moves to the second image 1007 side at a point in timeat which the second image 1007 is displayed. Further, when the gazingpoint 1006 deviates from the range to which the predetermined object1002 or the second image 1007 pertains or when the state in which thegazing point 1006 deviates is continued during a predetermined period oftime (for example, 3 seconds) or more, the display of the second image1007 ends (No in step 906), and the process returns to the detection ofthe predetermined object in step 902.

As described above, according to the third embodiment, in the wearableinformation display terminal 101, the display of the second image startsor ends according to the gazing point of the user on the relevantinformation of the predetermined object, and thus the display of theimage can be performed at a timing and a period of time in which a willor action of the user (for example, an action of drawing attention to aproduct=a will desiring to see information related to a product) isreflected. Similarly to the above embodiments, the user can use bothhands freely in the state in which the information display terminal 101is worn and need not operate the information display terminal 101 toselect an object, and thus convenience is improved. Since a situation inwhich the second image is displayed at an unnecessary timing or asituation in which information about a plurality of objects is displayedsimultaneously does not happen, there is an effect that a displayannoying the user can be prevented.

Fourth Embodiment

Next, a fourth embodiment of the information display terminal 101 willbe described with reference to FIG. 11 and FIGS. 12A and 12B. Aconfiguration of the information display terminal 101 is basically thesame as that illustrated in FIG. 1, and thus the description willproceed using the reference numerals allocated to the respectivecomponents of FIG. 1. In the present embodiment, similarly to the firstembodiment, the description will proceed with an example in which, whenthe user shops around a product in a store or the like, the informationdisplay terminal 101 displays the second image as relevant informationabout a predetermined object.

FIG. 11 is an operation flowchart according to the fourth embodiment.

FIGS. 12A and 12B are diagrams illustrating an example of the relevantinformation display according to the fourth embodiment, and illustratethe field of vision of the user.

Operations of step 1101 to step 1103 are the same as the operations ofstep 301 to step 303 in FIG. 3, and a detailed description thereof isomitted. When the process starts in step 1101, in step 1102, apredetermined object is detected, and in step 903, the relevantinformation related to the predetermined object is acquired from theInternet or the like. In the present embodiment, the description willproceed with an example in which a plurality of predetermined objectsare simultaneously detected in an imaging area in step 1102.

When the plurality of predetermined objects are detected, and therelevant information is acquired, in step 1104, a candidate mark isdisplayed. The candidate mark refers to a mark displayed when effectiverelevant information is acquired for the detected predetermined object.For example, FIG. 12A illustrates an example in which 1202 to 1205 in animaging area 1201 are detected as a predetermined object in step 1102,and effective relevant information is acquired for 1202, 1204, and 1205in step 1103. 1206 to 1208 are the candidate marks, the image processingunit 104 generates images as the candidate marks, and the displaycontrol unit 105 displays the candidate marks to overlap thecorresponding predetermined objects. The shape of the candidate mark isnot limited to the shape illustrated in FIG. 12A as long as thecandidate mark has the size that can be recognized by the user, and anassociated predetermined object can be determined.

Then, in step 1105, determination of a display trigger is performed. Inthis case, similarly to the above embodiments, a “hand” is registered asthe display trigger. In FIG. 12A, since there is no display trigger inthe imaging area 1201, only the candidate marks 1206, 1207, and 1208 aredisplayed, and the display of the second image related to the relevantinformation of each predetermined object is not performed.

On the other hand, as illustrated in FIG. 12B, when a specific object1209 serving as the display trigger is recognized in the imaging area1201 and detected to approach the predetermined object 1202, in step1106, the image processing unit 104 generates a second image 1210 as therelevant information of the predetermined object 1202 specified by thedisplay trigger, and the second image 1210 is displayed through thedisplay control unit 105 instead of the candidate mark 1206.

In this case, the predetermined object 1202 are close to the otherobjects 1203 to 1205. For this reason, the second image 1210 isdisplayed to overlap a part of the predetermined object 1202. Thus, anobject associated with the second image 1210 is explicitly displayed. Inthe other embodiments, in order to explicitly display an objectassociated with the second image, the second image may be displayed tooverlap a part of an associated object.

Similarly to the above embodiments, content displayed as the secondimage 1210 is the product information related to the predeterminedobject 1202, and for example, the reviews information or the marketprice information related to the product is displayed in the form easilyrecognized by the user. The displayed content is preferably informationuseful for the user and not particularly limited. The candidate marks1207 and 1208 indicate the state in which the associated objects are notselected, and thus the candidate marks are continuously displayedwithout change. A broken line of the imaging area 1201 is illustratedfor the sake of convenience and not displayed within the field of visionof the user.

In step 1107, the determination of the display trigger is continuouslyperformed, and the relevant information is displayed while the displaytrigger is being recognized (Yes in step 1107). Further, when theposition of the display trigger is changed, and a predetermined objectclose to the display trigger is changed, the candidate mark for thechanged predetermined object is replaced to the display of the secondimage related to the relevant information. Further, the predeterminedobject before the change is replaced from the second image related tothe relevant information to the candidate mark display. When thespecific object 1209 serving as the display trigger is hardly recognizedin the imaging area 1201 (No in step 1107), in step 1108, the secondimage display related to the relevant information ends, and the processreturns to the detection of the predetermined object in step 1102.However, the candidate mark displayed in the imaging area 1201 iscontinuously displayed.

As described above, according to the fourth embodiment, in the wearableinformation display terminal 101, the candidate mark is displayed beforethe relevant information display of the predetermined object, and thusthe user can recognize an object whose relevant information can bedisplayed in advance. For the relevant information, the display of thesecond image starts or ends using the display trigger, and thus thedisplay of the second image can be performed at a timing and a period oftime in which the will or action of the user is reflected. The user canuse both hands freely in the state in which the information displayterminal is worn and cause the display of the second image to beperformed while recognizing an object whose relevant information can bedisplayed. Further, the user need not operate the information displayterminal 101 to select an object, and thus convenience is improved.

As a modified example of the present embodiment, an embodiment in whichthe size or the color of the candidate marks 1206 to 1208 in FIG. 12A ischanged according to validity of content of the relevant information isconsidered. Since each relevant information is acquired at a point intime at which the candidate mark is displayed, for example, when anamount of additional information is large or when a product isdesignated by the user in advance, a display mark is enlarged or framedin a reddish color. As a result, the user can easily recognize it inadvance.

In the present embodiment, the example in which the operation ofdisplaying the candidate mark is performed in combination with the firstembodiment has been described, but the operation of displaying thecandidate mark may be performed in combination with the otherembodiments.

All the above embodiments have been described in connection with theexample in which, when the user shops around a product in a store,particularly, a clothing store, the information display terminaldisplays the relevant information about the predetermined object as thesecond image. However, the application scene is not limited thereto, andthe present invention can be applied to information display in differentcategories of stores or outdoors. The example in which the “hand” isused as the display trigger has been described, but any other things maybe designated as the display trigger.

The present invention is not limited to the above embodiments, andvarious modified examples are included. For example, the aboveembodiments have been described to facilitate understanding of thepresent invention, and the present invention is not limited to aconfiguration necessarily including all components described above. Allor some of the above-described functions or the processing units may beimplemented by hardware or may be implemented by software byinterpreting and executing a program for implementing the functionsthrough a processor. A control line or an information line is consideredto be necessary for description and illustrated, and all control linesor information lines are not necessarily illustrated.

REFERENCE SIGNS LIST

-   101, 504 Information display terminal-   102 Control unit-   103 Imaging unit-   104 Image processing unit-   105 Display control unit-   106 Audio input/output unit-   107 Audio processing unit-   108 Communication processing unit-   109 Communication input/output unit-   110 Position detecting unit-   111 Sensor input/output unit-   112 Various kinds of sensors-   201 User-   202, 401, 701, 801, 1001, 1201 Imaging area-   203 Image area-   402, 802, 1002, 1202 Predetermined object-   406, 805, 1007, 1210 Image of relevant information-   407, 804, 1209 Display trigger-   501 Internet-   502 Personal computer-   503 Mobile terminal-   505 Server-   506 Storage-   702 Registration frame-   1006 Gazing point-   1206, 1207, 1208 Candidate mark.

The invention claimed:
 1. A method for displaying information in afield-of-vision direction of a user on a head mounted display apparatus,the method comprising: imaging, by a camera provided on the head mounteddisplay apparatus, in the field-of-vision direction; and communicatingbetween the head mounted display apparatus and an information terminal;detecting, by the information terminal, a first object from a firstimage imaged by the camera; acquiring, by the information terminal,relevant information of the first object over a network; generating, bythe information terminal, a second image from the acquired relevantinformation; imaging a third image by the camera; designating, by theinformation terminal, a second object positioned in a predetermined areaof the third image; registering, by the information terminal,information identifying the second object as a trigger by which thesecond image is displayed; detecting, by the information terminal, thatthe second object has been imaged by the camera using the registeredinformation of the second object; and in response to the detected secondobject being determined to be at a position close to the first object,displaying, by the head mounted display apparatus, the second image. 2.The method for displaying information according to claim 1, furthercomprising: in response to the relevant information of the first objectbeing acquired over the network prior to the detection of the secondobject; generating, by the information terminal, a specific mark imagerelated to the first object; displaying, by the head mounted displayapparatus, the specific mark image to overlap a part of the firstobject; and in response to the detected second object being determinedto be at the position close to the first object, displaying, by the headmounted display apparatus, the second image in place of the specificmark image.
 3. The method for displaying information according to claim1, wherein the second image is displayed to overlap a part of the firstobject.
 4. The method for displaying information according to claim 1,further comprising: stopping, by the head mounted display apparatus, thedisplay of the second image in response to the detected second objectbeing determined to have moved away from the position close to the firstobject.